Spatio-temporal noise removal method using block classification and display device using the same

ABSTRACT

A spatio-temporal noise removal method using block classification that removes noise from a third field by using first, second, and third fields which are continuously inputted. The method includes generating first, second, and third motion-compensated fields; classifying blocks of the third field into a uniform region and a non-uniform region based on a variance value of a generated difference image; performing temporal filtering over every block of the third field; performing spatio-temporal filtering over every block of the first and third motion-compensated fields and the third field and performing the temporal filtering over the every block of the third field based on spatially-filtered value; and outputting a third noise-removed field by applying a weighted value to the temporal-filtered value and the spatio-temporal-filtered value.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit under 35 U.S.C. §119 from Korean PatentApplication 2005-51632, filed on Jun. 15, 2005, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a spatio-temporal noise removal methodusing block classification and a display device using the same forremoving spatio-temporal noise by performing spatio-temporal filteringdepending on whether a block in an image is a uniform region formed withsimilar pixel values.

2. Description of the Related Art

In the conventional image noise removal method, if a current field isinputted with noise, the conventional noise removal method removes thenoise from the current field based on motion compensation or motionestimation calculated by using a previous field from which noiseinputted prior to the current field has been removed.

In detail, the conventional image noise removal method calculates amotion vector between a current field and a noise-removed previousfield, compensates for motion, and uses motion estimation between thecurrent field and the noise-removed previous field. Themotion-compensated field and a current field for noise removal are usedfor the one-dimensional temporal filtering or the three-dimensionalspatio-temporal filtering. In this case, the finite impulse response(FIR) filter can be used at the time of the one-dimensional temporalfiltering, and the infinite impulse response (IIR) filter can be used atthe time of the three-dimensional spatio-temporal filtering.

However, at the time the temporal filter is used for noise removal,there exists a problem that noise appearing in space is not effectivelyremoved. Further, at the time the spatial filter is used for noiseremoval, there exists a flickering problem of varying a brightnessdifference between fields continuous in time since time-dependentrelationships between fields are not used.

Therefore, even a spatial filter for noise removal is required to usetime information, and a noise removal method is required to remove noiseby using a temporal filter and a spatial filter using time information,depending on a region of an image.

SUMMARY OF THE INVENTION

The present invention has been developed in order to solve the abovedrawbacks and other problems associated with the conventionalarrangement and other disadvantages not described above. Also, thepresent invention is not required to overcome the disadvantagesdescribed above, and an illustrative, non-limiting embodiment of thepresent invention may not overcome any of the problems described above.An aspect of the present invention is to provide a spatio-temporal noiseremoval method and a display device using the same, capable ofeffectively removing noise by classifying an image into a uniform regionhaving similar pixel values in a block of an image and a non-uniformregion and then performing spatio-temporal filtering over the uniformregion and temporal filtering over the non-uniform region.

The foregoing and other objects and advantages are substantiallyrealized by providing a display device for displaying images by removingnoise from a third field by using first, second, and third fields whichare continuously inputted, comprising a motion calculation part forgenerating a first motion-compensated field being a motion-compensatedfield by using the third field and the second noise-removed field, forgenerating a second motion-compensated field by using the first andsecond noise-removed fields, and for generating a thirdmotion-compensated field by using the second motion-compensated fieldand the third field; a classification part for classifying blocks of thethird field into a uniform region and a non-uniform region based on avariance value of a generated difference image by using the third fieldand any of the first and second motion-compensated fields; a temporalfilter for performing temporal filtering over every block of the thirdfield based on the first and third motion-compensated fields, the thirdfield, and the variance value; a spatio-temporal filter for performingspatio-temporal filtering by performing spatial filtering over everyblock of the first and third motion-compensated fields and the thirdfield and performing the temporal filtering over the every block of thethird field based on spatially-filtered value; and an arithmetic logicunit for outputting a third noise-removed field by applying a weightvalue to the temporal-filtered value and the spatio-temporal-filteredvalue, respectively, depending on whether a block in the third field isthe uniform region or the non-uniform region.

Preferably, but not necessarily, the motion calculation part includes afirst motion calculation unit for generating the firstmotion-compensated field being a motion-compensated field by using thethird field and the second noise-removed field; a secondmotion-compensated unit for generating the second motion-compensatedfield being a motion-compensated field by using the first and secondnoise-removed fields; and a third motion calculation unit for generatingthe third motion-compensated field being a motion-compensated field byusing the second motion-compensated field and the first noise-removedfield.

Preferably, but not necessarily, the classification part includes anadder for generating a difference image by calculating a pixel valuedifference between the third field and any of the first and secondmotion-compensated fields; a noise estimation unit for estimating asnoise a variance value having a maximum frequency after calculating avariance value over every block of the difference image; and a regionclassification unit for here, {circumflex over (f)}_(s−t)(x, y, k)denotes an output value of the spatio-temporal filter,f̂₁^(s)(x, y, k), f̂₂^(s)(x, y, k), and  f̂^(s)(x, y, k)denote spatially-filtered values over the first motion-compensatedfield, the third motion-compensated field, and the third field,respectively, and (x,y,k) denotes a pixel position.

Further, the temporal filter performing the temporal filtering overevery block of the third field by using an equation below:$\begin{matrix}{{{\hat{f}}_{t}\left( {x,y,k} \right)} = \frac{\sigma_{f}^{2}\left( {x,y,k} \right)}{{\sigma_{f}^{2}\left( {x,y,k} \right)} + {\sigma_{n}^{2}\left( {x,y,k} \right)}}} \\{\left\{ {{g\left( {x,y,k} \right)} - {E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack}} \right\} +} \\{{E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack},\quad{here},} \\{{E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack} = {\frac{1}{\quad 3}\left( {{g\left( {x,y,k} \right)} + {{\quad\hat{f}}_{1}^{\quad{mc}}\left( {x,y,k} \right)} +} \right.}} \\{\left. {{\hat{f}}_{2}^{\quad{mc}}\left( {x,y,k} \right)} \right),} \\{{\hat{f}}_{t}\left( {x,y,k} \right)}\end{matrix}$denotes the temporal-filtered value, g(x,y,k) the third field,f̂₁^(mc)(x, y, k)the first motion-compensated field, f̂₂^(mc)(x, y, k)the third motion-compensated field, (x,y,k) a pixel position in anarbitrary block, σ_(n) ² a variance value of the uniform region, andσ_(f) ² a variance value of a filtered block.

The spatio-temporal filter and the temporal filter are a Linear MinimumMean Square Error (LMMSE) filter.

The foregoing and other objects and advantages are substantiallyrealized by providing a spatio-temporal noise removal method using blockclassification for noise removal from a third field by using first,second, and third fields which are continuously inputted, comprisingsteps of generating a first motion-compensated field being amotion-compensated field by using the third field and the secondnoise-removed field, generating a second motion-compensated field byusing the first and second noise-removed fields, and generating a thirdmotion-compensated field by using the second motion-compensated fieldand the third field; classifying blocks of the third field into auniform region and a non-uniform region based on a variance value of agenerated difference image by using the third field and any of the firstand second motion-compensated fields; performing temporal filtering overevery block of the third field based on the first and thirdmotion-compensated fields, the third field, and the variance value;performing spatio-temporal filtering by performing spatial filteringover every block of the first and third motion-compensated fields andthe third field, and then performing the temporal filtering over theevery block of the third field based on spatially-filtered value; andoutputting a third noise-removed field by applying a weigh value to thetemporal-filtered value and the spatio-temporal-filtered value,respectively, depending on whether a block in the third field is theuniform region or the non-uniform region.

Preferably, but not necessarily, the step of generating themotion-compensated field includes steps of generating the firstmotion-compensated field being a motion-compensated field by using thethird field and the second noise-removed field; generating the secondmotion-compensated field being a motion-compensated field by using thefirst and second noise-removed fields; and generating the thirdmotion-compensated field being a motion-compensated field by using thesecond motion-compensated field and the first noise-removed field.

Preferably, but not necessarily, the classification step includes stepsof generating a difference image by calculating a pixel value differencebetween the third field and any of the first and secondmotion-compensated fields; estimating as noise a variance value having amaximum frequency after calculating a variance value over every block ofthe difference image; and classifying as the uniform region a blockhaving a variance value larger than the estimated noise.

Preferably, but not necessarily, the spatio-temporal noise removalmethod further comprises a step of generating a weight value in orderfor an output value of the spatio-temporal filter to be taken into moreaccount than an output value of the temporal filter if an arbitraryblock of the third field is the uniform region, and generating a weightvalue in order for the output value of the temporal filter to be takeninto more account than the output value of the spatio-temporal filter.

Further, the third noise-removed field is calculated by an equation asbelow:{circumflex over (f)}(x,y,k)=w×f _(t)(x,y,k)+(1−w)×f _(s−t)(x,y,k),in here, {circumflex over (f)}(x,y,k) denotes the third noise-removedfield, w and (1−w) denote a weight value applied to a temporal-filteredvalue and a weight value applied to a spatio-temporal-filtered value,respectively, f_(t)(x,y,k) denotes a temporal-filtered value, andf_(s−t)(x,y,k) denotes a spatio-temporal-filtered value.

Further, the step of performing the spatio-temporal filtering performsthe spatio-temporal filtering over every block of the third field byusing an equation as below:${{{\hat{f}}_{s - t}\left( {x,y,k} \right)} = {\frac{1}{3}\left\{ {{{\hat{f}}_{1}^{s}\left( {x,y,k} \right)} + {{\hat{f}}_{2}^{s}\left( {x,y,k} \right)} + {{\hat{f}}^{s}\left( {x,y,k} \right)}} \right\}}},$in here, {circumflex over (f)}_(s−t)(x,y,k), denotes aspatio-temporal-filtered value,f̂₁^(s)(x, y, k),  f̂₂^(s)(x, y, k),  and  f̂^(s)(x, y, k)denote spatially-filtered values over the first motion-compensatedfield, the third motion-compensated field, and the third field,respectively, and (x,y,k) denotes a pixel position.

Further, the step of performing the temporal filtering performs thetemporal filtering over every block of the third field by using anequation as below: $\begin{matrix}{{{\hat{f}}_{t}\left( {x,y,k} \right)} = \frac{\sigma_{f}^{2}\left( {x,y,k} \right)}{{\sigma_{f}^{2}\left( {x,y,k} \right)} + {\sigma_{n}^{2}\left( {x,y,k} \right)}}} \\{\left\{ {{g\left( {x,y,k} \right)} - {E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack}} \right\} +} \\{{E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack},\quad{{in}\quad{here}},} \\{{E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack} = {\frac{1}{\quad 3}\left( {{g\left( {x,y,k} \right)} + {{\quad\hat{f}}_{1}^{\quad{mc}}\left( {x,y,k} \right)} +} \right.}} \\{\left. {{\hat{f}}_{2}^{\quad{mc}}\left( {x,y,k} \right)} \right),} \\{{\hat{f}}_{t}\left( {x,y,k} \right)}\end{matrix}$denotes the temporal-filtered value, g(x,y,k) the third field,f̂₁^(mc)(x, y, k)the first motion-compensated field, f̂₂^(mc)(x, y, k)the third motion-compensated field, (x,y,k) a pixel position in anarbitrary block, σ_(n) ² a variance value of the uniform region, andσ_(f) ² a variance value of a filtered block.

BRIEF DESCRIPTION OF THE DRAWINGS

The above aspects and features of the present invention will be moreapparent by describing exemplary embodiments of the present inventionwith reference to the accompanying drawings.

FIG. 1 is a block diagram for showing a display device using aspatio-temporal noise removal method using block classificationaccording to an embodiment of the present invention.

FIG. 2 is a block diagram for showing a classification part of FIG. 1.

FIG. 3 is a flow chart for explaining a spatio-temporal noise removalmethod using the block classification according to an embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Hereinafter, the present invention will be described in detail withreference to the accompanying drawings.

FIG. 1 is a block diagram for showing a display device using aspatio-temporal noise removal method using block classificationaccording to an embodiment of the present invention. Further, FIG. 2 isa block diagram for showing the classification part 200 of FIG. 1.

A display device using a spatio-temporal noise removal method usingblock classification according to the present invention removes noisefrom a third field by using the first, second, and third fieldscontinuously inputted in time. Here, the first and second fields arenoise-removed fields, and the third filed is a noise-carrying field. InFIG. 1, g(x,y,k) denotes a noise-carrying third field, {circumflex over(f)}(x,y,k−1) denotes a noise-removed second field (or secondnoise-removed field), and {circumflex over (f)}(x,y,k−2) denotes anoise-removed first field (or first noise-removed field).

In FIG. 1, the display device using the spatio-temporal noise removalmethod using block classification according to the present invention hasa first motion calculation unit 110, a second motion calculation unit120, a third motion calculation unit 130, a classification part 200, aweight value unit 300, a spatio-temporal filter 400, a temporal filter400, and an arithmetic logic unit 600.

The first, second, and third motion calculation units 110, 120, and 130generate motion-compensated fields by using a first noise-removed field,a second noise-removed field, and a third field. The first motioncalculation unit 110 generates a motion-compensated field, that is, afirst motion-compensated field, f̂₁^(mc)(x,  y,  k),by using the third field for noise removal and a noise-removed secondfield. Here, the first motion-compensated field corresponds to a fieldidentical in time to the third field.

Further, the second motion calculation unit 120 generates amotion-compensated field, that is, a second motion-compensated field,${{\hat{f\quad}}_{1}^{mc}\left( {x,y,{k - 1}} \right)},$by using the first noise-removed field and the second noise-removedfield. The second motion-compensated field corresponds to a fieldidentical in time to the second field. Further, the third motioncalculation unit 130 generates a motion-compensated field, that is, athird motion-compensated field, f̂₂^(mc)(x, y, k),by using the second motion-compensated field and the thirdnoise-carrying field. Here, the third motion-compensated fieldcorresponds to a field identical in time to the third field as in thefirst motion-compensated field.

The first and second noise-removed fields used in the first, second, andthird motion calculation units 110, 120, and 130 are output values ofthe arithmetic logic unit 600, which are noise-removed fields accordingto the present invention.

The classification part 200 classifies every block of the third fieldinto a uniform region having similar pixel values in a block and anon-uniform region having non-similar pixel values in a block, by usingthe third field and any of the first and second motion-compensatedfields. Description is made in detail on operations of theclassification part 200. That is, the classification part 200 has anadder 210, a noise estimation unit 220, and a region classification unit230. The adder 210 generates a difference image by calculating a pixelvalue difference between the third field and the firstmotion-compensated field being an output value of the first motioncalculation unit 110. Additionally, the adder 210 can generate adifference image by using the third motion-compensated field being anoutput value of the third motion calculation unit 130 and the thirdfield. Hereinafter, description will be made on how the firstmotion-compensated field is used, for an example.

The adder 210 can generate a difference image by using Equation 1 asbelow. $\begin{matrix}{{{d\left( {x,y,k} \right)} = {{g\left( {x,y,k} \right)} - {{\hat{f}}_{1}^{mc}\left( {x,y,k} \right)}}},} & \left\lbrack {{Equation}\quad 1} \right\rbrack\end{matrix}$in here, d(x,y,k) denotes a difference image as an output value of theadder 210, g(x,y,k) denotes a third noise-carrying field,${\hat{f\quad}}_{1}^{mc}\left( {x,y,k} \right)$denotes a first motion-compensated field, and (x,y,k) denotes a pixelposition. As shown in Equation 1, a difference image can be generatedthrough calculating a difference between a pixel value of the thirdfield and a pixel value of the first motion-compensated field.

The difference image contains noise and motion errors. The differenceimage is generated through calculating a pixel value difference betweenthe third field and the first motion-compensated field which isidentical in time to the third field and generated by using the secondand third noise-removed fields, so the non-uniform region such as anedge region of the third field can contain noise as well as motionerrors.

The noise estimation unit 220 divides a difference image generated fromthe adder 210 into blocks of a certain size, calculates a variance valueover every block, and estimates noise, {circumflex over (σ)}_(n)(k),based on the calculated variance values. The noise estimation unit 220generates a histogram based on the calculated variance values, andestimates as noise a variance value having the maximum frequency. Sincean image is mostly formed with uniform regions, a region having variancevalues lower than a variance value having the maximum frequency can bedecided as a uniform region. The uniform region in a difference imagedoes not contain motion errors, unlike a non-uniform region such as anedge region, but contains noise. Thus, the noise estimation unit 220estimates as noise the variance values of a uniform region containingthe variance value having the maximum frequency.

The region classification unit 230 compares a variance value calculatedover each block with estimated noise, and classifies every block intothe uniform region and the non-uniform region. Noise, as well as a blockof a difference image containing motion errors, can have a relativelyhigh variance value, and a block of a difference image containing onlynoise has a low variance value. Thus, the region classification unit 230can classify a block having a variance value higher than the estimatednoise into non-uniform region such as an edge region having non-similarpixel values in the block, and can classify a block having variancevalues lower than the estimated noise into a uniform region containingonly noise.

The weight value unit 300 inputs noise estimated by the classificationpart 200 and a variance value σij calculated over each block, andgenerates a weight value w to be applied to the spatio-temporal filter400 and the temporal filter 500. The weight value unit 300 generates aweight value in order for relatively a high weight value to be set to anoutput value of the spatio-temporal filter 400 if a block of the thirdfield is a uniform region, and, if a block of the third field is anon-uniform region, generates a weight value in order for relatively ahigh weight value to be set to an output value of the temporal filter500.

Specifically, if a variance value calculated over every block of adifference image is lower than the estimated noise, that is, if anarbitrary block of the third field is a uniform region, the weight valueunit 300 generates a weight value w of 0. However, if the calculatedvariance value is two times higher than the estimated noise, that is, ifan arbitrary block of the third field is a non-uniform region, theweight value unit 300 generates a weight value w of 1.

Further, if a variance value calculated over every block of a differenceimage is higher than estimated noise or lower than twice the estimatednoise, the weight value unit 300 decides a weight value w of which valueis proportional to a variance value calculated over every block. Thatis, the weight value w is decided to be closer to 0 as the variancevalue calculated over every block has a value similar to the estimatednoise, and the weight value w is decided to be closer to 1 as thevariance value calculated over every block has a value similar to twicethe estimated noise.

The spatio-temporal filter 400 carries out the spatial filtering andthen the temporal filtering over the first motion-compensated field, thethird motion-compensated field, and the third field, respectively, basedon the spatially-filtered values, so as to carry out the spatio-temporalfiltering. In here, the spatio-temporal filter 400 can be a LinearMinimum Mean Square Error (LMMSE) filter. The spatio-temporal filter 400carries out the spatial filtering over the first motion-compensatedfield, the third motion-compensated field, and the third field,respectively, adds the spatially-filtered values thereof, and carriesout the spatio-temporal filtering over every block of the third field.

The temporal filter 500 performs the temporal filtering by using avariance value calculated over every block of the firstmotion-compensated field, third motion-compensated field, and thirdfield, thereby removing noise from the third field. In here, thetemporal filter 500 can be the Linear Minimum Mean Square Error (LMMSE)filter.

The arithmetic logic unit 600 applies a weight value of the weight valueunit 300 to an output value of the spatio-temporal filter 400 and anoutput value of the temporal filter 500, respectively, and adds theweight value-applied output values of the spatio-temporal filter 400 andthe temporal filter 500. Thus, the arithmetic logic unit 600 can expressthe noise-removed third field, which is an output value, in Equation 2as below.{circumflex over (f)}(x,y,k)=w×f _(t)(x,y,k)+(1−w)×f _(s−t)(x,y,k),  [Equation 2]in here, {circumflex over (f)}(x,y,k) denotes a third noise-removedfield which is an output value of the arithmetic logic unit 600, and wand (1−w) denote a weight value set to an output value of the temporalfilter 500 and a weight value set to an output value of thespatio-temporal filter 400, respectively. Further, f_(t)(x,y,k) denotesan output value of the temporal filter 500, f_(s−t)(x,y,k) denotes anoutput value of the spatio-temporal filter 400, and (x,y,k) denotes apixel position.

As expressed in Equation 2, the third noise-removed field calculated bythe arithmetic logic unit 600 is a soft-switching value of an outputvalue of the temporal filter 500 and an output value of thespatio-temporal filter 400. If a block of the third field is a uniformregion, a weight value of 0 is applied to the output value of thespatio-temporal filter 400 and a weight value of 1 is applied to theoutput value of the temporal filter 500, so the output value of thetemporal filter 500 becomes an output value of the arithmetic logic unit600.

However, if a block of the third field is a non-uniform region, a weightvalue to an output value of the spatio-temporal filter 400 becomes closeto ‘1’, and a weight value applied to an output value of the temporalfilter 500 becomes close to ‘0’. Thus, the arithmetic logic unit 600outputs an addition value of an output value of the temporal filter 500and an output value of the spatio-temporal filter 400 taking an outputvalue of the spatio-temporal filter 400 into more account than an outputvalue of the temporal filter 500.

The third noise-removed field outputted by the arithmetic logic unit 600is used for noise removal from fourth and fifth noise-carrying fieldswhich are subsequently inputted.

FIG. 3 is a flow chart for explaining a spatio-temporal noise removalmethod using block classification according to an embodiment of thepresent invention. Description will be made on a method for removingnoise from the third field, using the first, second, and third fieldswhich are continuously inputted, for an example.

In FIG. 3, the first, second, and third fields, which are continuouslyinputted, are used for generation of a motion-compensated field (S901).In here, the first and second fields are noise-removed fields, and thethird field is a field for noise removal. The third field and the secondnoise-removed field are used for generation of the firstmotion-compensated field identical in time to the third field which is amotion-compensated field.

Further, the first and second noise-removed fields are used forgeneration of the second motion-compensated field identical in time tothe second field which is a motion-compensated field, and the secondmotion-compensated field and the third noise-carrying field are used forgeneration of the third motion-compensated field identical in time tothe third field which is a motion-compensated field.

Next, a generated motion-compensated image is used for classification ofthe blocks of the third field into a uniform region having similar pixelvalues in a block and a non-uniform region having non-similar pixelvalues (S903). Specifically, the motion-compensated image can be thefirst motion-compensated field or the third motion-compensated field.Hereinafter, description will be made on classification in which thefirst motion-compensated field and the third field are used forclassification of the blocks of the third field into the uniform regionand the non-uniform region.

Pixel value differences between the first motion-compensated field andthe third field are calculated for generation of a difference image. Inhere, the difference image can contain noise and motion errors of thethird field. Out of the difference image, the uniform region, which hasno motion errors, contains only noise.

Equation 3 as below can be used to calculate the difference image.$\begin{matrix}{{{d\left( {x,y,k} \right)} = {{g\left( {x,y,k} \right)} - {{\hat{f}}_{1}^{mc}\left( {x,y,k} \right)}}},} & \left\lbrack {{Equation}\quad 3} \right\rbrack\end{matrix}$here, d(x,y,k) denotes a difference image, g(x,y,k) denotes the thirdfield, and f̂₁^(mc)(x, y, k)denotes the first motion-compensated field, and (x,y,k) denotes a pixelposition.

Further, a variance value of every block is calculated out of thedifference image shown in Equation 3, and noise contained in the thirdfield is estimated. A histogram with the variance value calculated forevery block is generated in which the variance value having the maximumfrequency is estimated as noise. Since it can be considered that mostregions of one field correspond to a uniform region, the variance valuehaving the maximum frequency is decided to be a variance value of theuniform region. Since the uniform region of a difference image does notcontain motion errors but only noise, the variance value having themaximum frequency is estimated to be noise of the third field.

Further, the estimated noise and the variance value over every block arecompared to each other, blocks having variance values higher than theestimated noise are classified into the non-uniform region, and blockshaving variance values lower than the estimated noise are classifiedinto the uniform region. The block having a variance value higher thanthe estimated noise is a block having motion errors as a region such asedges, which can be decided as the non-uniform block.

Next, the spatio-temporal filtering and the temporal filtering areperformed over the third field (S905). Every block of the third field isfiltered. Since the pixel values in an arbitrary block are similar toone another when the block is the uniform region, the information on thethird field and the information on the first and second fields are usedtogether for the spatio-temporal filtering over the uniform region,which can reduce flickering. However, since pixel values in an arbitraryblock are not similar when the block is the non-uniform region, thefirst and second field information is used for filtering over thenon-uniform region, which can reduce flickering. Thus, a weight value isassigned to a spatio-temporal-filtered value and a temporal-filteredvalue depending on whether an arbitrary block is the uniform region orthe non-uniform region, which can reduce noise from the third field forenhancement of a noise removal effect.

Equation 4 can be used for the spatio-temporal filtering for noiseremoval from an arbitrary block of the third field. $\begin{matrix}{{{{\hat{f}}_{s - t}\left( {x,y,k} \right)} = {\frac{1}{3}\left\{ {{{\hat{f}}_{1}^{s}\left( {x,y,k} \right)} + {{\hat{f}}_{2}^{s}\left( {x,y,k} \right)} + {{\hat{f}}^{s}\left( {x,y,k} \right)}} \right\}}},} & \left\lbrack {{Equation}\quad 4} \right\rbrack\end{matrix}$here, f̂_(s − t)(x, y, k)denotes an output value of the spatio-temporal filter,${{\hat{f}}_{1}^{s}\left( {x,y,k} \right)},{{\hat{f}}_{2}^{s}\left( {x,y,k} \right)}\quad,\quad{{and}{\hat{\quad f}}^{s}\left( {x,y,k} \right)}$denote spatially-filtered values over arbitrary blocks of the firstmotion-compensated field, the third motion-compensated field, and thethird field, respectively, and (x,y,k) denotes a pixel position of anarbitrary block of the first motion-compensated field, the thirdmotion-compensated field, and the third field.

Further, Equation 5 below can be used for temporal filtering for noiseremoval over an arbitrary block of the third field. $\begin{matrix}{{{{\hat{f}}_{t}\left( {x,y,k} \right)} = {{\frac{\sigma_{f}^{2}\left( {x,y,k} \right)}{{\sigma_{f}^{2}\left( {x,y,k} \right)} + {\sigma_{n}^{2}\left( {x,y,k} \right)}}\left\{ {{g\left( {x,y,k} \right)} - {E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack}} \right\}} + {E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack}}},{here},\text{}{{E\left\lbrack {g\left( {x,y,k} \right)} \right\rbrack} = {\frac{1}{3}\left( {{g\left( {x,y,k} \right)} + {{\hat{f}}_{1}^{mc}\left( {x,y,k} \right)} + {{\hat{f}}_{2}^{mc}\left( {x,y,k} \right)}} \right)}},{{\hat{f}}_{t}^{\quad}\left( {x,y,k} \right)}} & \left\lbrack {{Equation}\quad 5} \right\rbrack\end{matrix}$denotes a temporal-filtered value, g(x,y,k) the third field,f̂₁^(mc)(x,  y,  k)the first motion-compensated field, f̂₂^(mc)(x,  y,  k)the third motion-compensated field, (x,y,k) a pixel position in anarbitrary block, σ_(n) ² estimated noise, and σ_(f) ² a variance valueof a filtered block.

Next, the estimated noise and a variance value calculated over everyblock of a difference image are used for generation of a weight value tobe applied to a spatiotemporal-filtered value and a temporal-filteredvalue, and the third noise-removed field is outputted (S907).

If a block in the third field for noise removal is classified as auniform region, a weight value is generated in order for thespatio-temporal-filtered value to be relatively more considered, and, ifclassified as a non-uniform region, a weight value is generated in orderfor the temporal-filtered value to be relatively more considered.

Further, Equation 6 below can be used to calculate the thirdnoise-removed field.{circumflex over (f)}(x,y,k)=w×f _(t)(x,y,k)+(1−w)×f _(s−t)(x,y,k),  [Equation 6]in here, {circumflex over (f)}(x,y,k) denotes the third noise-removedfield, w and (1−w) denote a weight value applied to a temporal-filteredvalue and a weight value applied to a spatio-temporal-filtered value,respectively, f_(t)(x,y,k) denotes a temporal-filtered value,f_(s−t)(x,y,k) denotes a spatio-temporal-filtered value, and (x,y,k) apixel position.

Thus, if a variance value calculated over every block of a differenceimage is lower than the estimated noise, that is, if the arbitrary blockof the third field is a uniform region, the weight value becomes a 0, sothat only a spatio-temporal-filtered value is outputted.

However, if a variance value calculated over every block of a differenceimage is higher than twice the estimated noise, that is, if an arbitraryblock of the third field is a non-uniform region, the weight valuebecomes a 1, so only a temporal-filtered value is outputted.

If a variance value calculated over every block of a difference image ishigher than the estimated noise but lower than twice the estimatednoise, a weight value is decided to be proportional to a variance valuecalculated over every block. That is, as a variance value calculatedover every block has a value more similar to the estimated noise, aweight value is decided to be closer to 0, and, as a variance valuecalculated over every block has a value more similar to twice theestimated noise, a weight value is decided to be closer to 1.

Therefore, depending on whether an arbitrary block is a uniform regionafter a noise-carrying field is divided into blocks, the spatio-temporalfiltering or the temporal filtering is performed, and aspatio-temporal-filtered value and a temporal-filtered value are added,so noise can be more effectively removed than when only the temporalfiltering is performed or only the spatio-temporal filtering isperformed.

As aforementioned, the present invention can adaptively perform thespatiotemporal filtering depending on a region of an input image,thereby effectively removing noise contained in the image.

The foregoing embodiments and advantages are merely exemplary and arenot to be construed as limiting the present invention. The presentteaching can be readily applied to other types of apparatuses. Also, thedescription of the embodiments of the present invention is intended tobe illustrative, and not to limit the scope of the claims, and manyalternatives, modifications, and variations will be apparent to thoseskilled in the art.

1. A display device for displaying images by removing noise from a thirdfield by using a first noise-removed field, a second noise-removedfield, and the third field, comprising: a motion calculation partadapted to generate a first motion-compensated field by using the thirdfield and the second noise-removed field, generate a secondmotion-compensated field by using the first and second noise-removedfields, and generate a third motion-compensated field by using thesecond motion-compensated field and the third field; a classificationpart adapted to generate blocks of the third field into a uniform regionand a non-uniform region based on a variance value of a generateddifference image by using the third field and any of the first andsecond motion-compensated fields; a temporal filter adapted to performtemporal filtering over every block of the third field based on thefirst and third motion-compensated fields, the third field, and thevariance value; a spatio-temporal filter adapted to performspatio-temporal filtering by performing spatial filtering over everyblock of the first and third motion-compensated fields and the thirdfield, and by performing the temporal filtering over every block of thethird field based on the spatially-filtered value; and an arithmeticlogic unit adapted to output a third noise-removed field by applying aweighted value to the temporal-filtered value and thespatio-temporal-filtered value, respectively, depending on whether ablock in the third field is the uniform region or the non-uniformregion.
 2. The display device as claimed in claim 1, wherein the motioncalculation part comprises: a first motion calculation unit adapted togenerate the first motion-compensated field by using the third field andthe second noise-removed field; a second motion-compensated unit adaptedto generate the second motion-compensated field by using the first andsecond noise-removed fields; and a third motion calculation unit adaptedto generate the third motion-compensated field by using the secondmotion-compensated field and the first noise-removed field.
 3. Thedisplay device as claimed in claim 1, wherein the classification partincludes: an adder adapted to generate a difference image by calculatinga pixel value difference between the third field and any of the firstand second motion-compensated fields; a noise estimation unit adapted toestimate, as noise, a variance value having a maximum frequency aftercalculating a variance value over every block of the difference image;and a region classification unit adapted to classify, as the uniformregion, a block having a variance value higher than the estimated noise.4. The display device as claimed in claim 1, further comprising a weightvalue unit adapted to generate a weight value in order for an outputvalue of the spatio-temporal filter to be taken into more account thanan output value of the temporal filter if an arbitrary block of thethird field is the uniform region, and generate a weight value in orderfor the output value of the temporal filter to be taken into moreaccount than the output value of the spatio-temporal filter.
 5. Thedisplay device as claimed in claim 1, wherein the arithmetic logic unitoutputs the third noise-removed field by using an equation as below:{circumflex over (f)}(x,y,k)=w×f _(t)(x,y,k)+(1−w)×f _(s−t)(x,y,k),here, {circumflex over (f)}(x,y,k) denotes the third noise-removedfield, w and (1−w) denote a weight value applied to a temporal-filteredvalue and a weight value applied to a spatio-temporal-filtered value,respectively, f_(t)(x,y,k) denotes an output value of the temporalfilter, and f_(s−t)(x,y,k) denotes an output value of thespatio-temporal filter.
 6. The display device as claimed in claim 1,wherein the spatio-temporal filter performs the spatio-temporalfiltering over every block of the third field by using an equation asbelow:${{{\hat{f}}_{s - t}\left( {x,y,k} \right)} = {\frac{1}{3}\left\{ {{{\hat{f}}_{1}^{s}\left( {x,y,k} \right)} + {{\hat{f}}_{2}^{s}\left( {x,y,k} \right)} + {{\hat{f}}^{s}\left( {x,y,k} \right)}} \right\}}},$here, {circumflex over (f)}_(s−t)(x,y,k) denotes an output value of thespatio-temporal filter,f̂₁^(s)(x, y, k), f̂₂^(s)(x, y, k), and  f̂^(s)(x, y, k) denotespatially-filtered values over the first motion-compensated field, thethird motion-compensated field, and the third field, respectively, and(x,y,k) denotes a pixel position.
 7. The display device as claimed inclaim 1, wherein the temporal filter performs the temporal filteringover every block of the third field by using an equation as below:$\begin{matrix}{{{\hat{f}}_{t}\left( {x,\quad y,\quad k} \right)}\quad = \frac{\sigma_{f}^{2}\left( {x,\quad y,\quad k} \right)}{{\sigma_{f}^{2}\left( {x,\quad y,\quad k} \right)}\quad + \quad{\sigma_{n}^{2}\left( {x,\quad y,\quad k} \right)}}} \\{{\left\{ {{g\quad\left( {x,\quad y,\quad k} \right)}\quad - \quad{E\left\lbrack {g\quad\left( {x,\quad y,\quad k} \right)} \right\rbrack}} \right\} +}\quad} \\{{E\left\lbrack {g\left( {x,\quad y,\quad k} \right)} \right\rbrack},\quad{here},} \\{{E\left\lbrack {g\quad\left( {x,\quad y,\quad k} \right)} \right\rbrack} = {\frac{1}{\quad 3}\left( {{g\quad\left( {x,\quad y,\quad k} \right)} + {{\quad\hat{f}}_{1}^{\quad{mc}}\left( {x,\quad y,\quad k} \right)} +} \right.}} \\{\left. {{\hat{f}}_{2}^{\quad{mc}}\left( {x,\quad y,\quad k} \right)} \right),} \\{{\hat{f}}_{t}\left( {x,\quad y,\quad k} \right)}\end{matrix}$ denotes the temporal-filtered value, g(x,y,k) the thirdfield, f̂₁^(mc)(x,  y,  k) the first motion-compensated field,f̂₂^(mc)(x,  y,  k) the third motion-compensated field, (x,y,k) a pixelposition in an arbitrary block, σ_(n) ², a variance value of the uniformregion, and σ_(f) ² a variance value of a filtered block.
 8. The displaydevice as claimed in claim 1, wherein the spatio-temporal filter and thetemporal filter are a Linear Minimum Mean Square Error (LMMSE) filter.9. A spatio-temporal noise removal method using block classification fornoise removal from a third field by using a first noise-removed field, asecond noise-removed field, and the third field, comprising: generatinga first motion-compensated field by using the third field and the secondnoise-removed field, generating a second motion-compensated field byusing the first and second noise-removed fields, and generating a thirdmotion-compensated field by using the second motion-compensated fieldand the third field; classifying blocks of the third field into auniform region and a non-uniform region based on a variance value of agenerated difference image by using the third field and any of the firstand second motion-compensated fields; performing temporal filtering overevery block of the third field based on the first and thirdmotion-compensated fields, the third field, and the variance value;performing spatio-temporal filtering by performing spatial filteringover every block of the first and third motion-compensated fields andthe third field and performing the temporal filtering over the everyblock of the third field based on the spatially-filtered value; andoutputting a third noise-removed field by applying a weighted value tothe temporal-filtered value and the spatio-temporal-filtered value,respectively, depending on whether a block in the third field is theuniform region or the non-uniform region.
 10. The spatio-temporal noiseremoval method as claimed in claim 9, wherein generating themotion-compensated field comprises: generating the firstmotion-compensated field by using the third field and the secondnoise-removed field; generating the second motion-compensated field byusing the first and second noise-removed fields; and generating thethird motion-compensated field being a motion-compensated field by usingthe second motion-compensated field and the third field.
 11. Thespatio-temporal noise removal method as claimed in claim 9, whereinclassifying blocks of the third field comprises: generating a differenceimage by calculating a pixel value difference between the third fieldand any of the first and second motion-compensated fields; estimating asnoise a variance value having a maximum frequency after calculating avariance value over every block of the difference image; and classifyingas the uniform region a block having a variance value larger than theestimated noise.
 12. The spatio-temporal noise removal method as claimedin claim 9, further comprising: generating a weight value in order foran output value of the spatio-temporal filter to be taken into moreaccount than an output value of the temporal filter if an arbitraryblock of the third field is the uniform region, and generating a weightvalue in order for the output value of the temporal filter to be takeninto more account than the output value of the spatio-temporal filter.13. The spatio-temporal noise removal method as claimed in claim 9,wherein the third noise-removed field is calculated by an equation asbelow:{circumflex over (f)}(x,y,k)=w×f _(t)(x,y,k)+(1−w)×f _(s−t)(x,y,k), inhere, {circumflex over (f)}(x,y,k) denotes the third noise-removedfield, w and (1−w) denote a weight value applied to a temporal-filteredvalue and a weight value applied to a spatio-temporal-filtered value,respectively, f_(t)(x,y,k) denotes a temporal-filtered value, andf_(s−t)(x,y,k) denotes a spatio-temporal-filtered value.
 14. Thespatio-temporal noise method as claimed in claim 9, wherein inperforming the spatio-temporal filtering over every block of the thirdfield, an equation, as below, is used:{circumflex over (f)}(x,y,k)=w×f _(t)(x,y,k)+(1−w)×f _(s−t)(x,y,k),here, {circumflex over (f)}_(s−t)(x,y,k) denotes aspatio-temporal-filtered value,f̂₁^(s)(x, y, k), f̂₂^(s)(x, y, k), and  f̂^(s)(x, y, k) denotespatially-filtered values over the first motion-compensated field, thethird motion-compensated field, and the third field, respectively, and(x,y,k) denotes a pixel position.
 15. The spatio-temporal noise removalmethod as claimed in claim 9, wherein in performing the temporalfiltering over every block of the third field, an equation, as below, isused: $\begin{matrix}{{{\hat{f}}_{t}\left( {x,\quad y,\quad k} \right)}\quad = \frac{\sigma_{f}^{2}\left( {x,\quad y,\quad k} \right)}{{\sigma_{f}^{2}\left( {x,\quad y,\quad k} \right)}\quad + \quad{\sigma_{n}^{2}\left( {x,\quad y,\quad k} \right)}}} \\{{\left\{ {{g\quad\left( {x,\quad y,\quad k} \right)}\quad - \quad{E\left\lbrack {g\quad\left( {x,\quad y,\quad k} \right)} \right\rbrack}} \right\} +}\quad} \\{{E\left\lbrack {g\left( {x,\quad y,\quad k} \right)} \right\rbrack},\quad{here},} \\{{E\left\lbrack {g\quad\left( {x,\quad y,\quad k} \right)} \right\rbrack} = {\frac{1}{\quad 3}\left( {{g\quad\left( {x,\quad y,\quad k} \right)} + {{\quad\hat{f}}_{1}^{\quad{mc}}\left( {x,\quad y,\quad k} \right)} +} \right.}} \\{\left. {{\hat{f}}_{2}^{\quad{mc}}\left( {x,\quad y,\quad k} \right)} \right),} \\{{\hat{f}}_{t}\left( {x,\quad y,\quad k} \right)}\end{matrix}$ denotes the temporal-filtered value, g(x,y,k) the thirdfield, f̂₁^(mc)(x,  y,  k) the first motion-compensated field,f̂₂^(mc)(x, y, k) the third motion-compensated field, (x,y,k) a pixelposition in an arbitrary block, σ_(n) ² a variance value of the uniformregion, and σ_(f) ² a variance value of a filtered block.